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Abstract 


Flight research has shown the effectiveness of adaptive flight controls for improving aircraft 
safety and performance in the presence of uncertainties. The National Aeronautics and Space 
Administration’s (NASA)’s Integrated Resilient Aircraft Control (I RAC) project designed and 
conducted a series of flight experiments to study the impact of variations in adaptive controller 
design complexity on performance and handling qualities. A novel complexity metric was 
devised to compare the degrees of simplicity achieved in three variations of a model reference 
adaptive controller (MRAC) for NASA’s F-18 (McDonnell Douglas, now The Boeing Company, 
Chicago, Illinois) Full-Scale Advanced Systems Testbed (Gen-2A) aircraft. The complexity 
measures of these controllers are also compared to that of an earlier MRAC design for NASA’s 
Intelligent Flight Control System (IFCS) project and flown on a highly modified F-15 aircraft 
(McDonnell Douglas, now The Boeing Company, Chicago, Illinois). Pilot comments during the 
IRAC research flights pointed to the importance of workload on handling qualities ratings for 
failure and damage scenarios. Modifications to existing pilot aggressiveness and duty cycle 
metrics are presented and applied to the IRAC controllers. Finally, while adaptive controllers 
may alleviate the effects of failures or damage on an aircraft’s handling qualities, they also have 
the potential to introduce annoying changes to the flight dynamics or to the operation of aircraft 
systems. A nuisance rating scale is presented for the categorization of nuisance side-effects of 
adaptive controllers. 
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Acronyms 

ACAT 

ESA 

FAST 

Gen-1 

Gen-2 

Gen-2A 

IFCS 

IRAC 

MRAC 

NASA 

NDI 

on MRAC 
onMRAC+ 
PIO 
RFI 

sMRAC 

SOHO 


Autonomous Collision Avoidance Technology 

European Space Agency 

full-scale advanced systems testbed 

first-generation IFCS adaptive flight control system 

second-generation IFCS adaptive flight control system 

update to the second-generation IFCS adaptive flight control system 

intelligent flight control system 

integrated resilient aircraft control 

model reference adaptive control 

National Aeronautics and Space Administration 

nonlinear dynamic inversion 

MRAC with normalization and optimal control modification 
onMRAC with an additional adaptive parameter 
pilot-in-the-loop oscillation 
request for information 
simple MRAC 

Solar Heliospheric Observatory 


Mathematical Symbols 

B s matrix of input coefficients 

C error feedback compensator 

f A estimate of applied aerodynamic and inertial moments 
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reference model 
Laplace operator 
time 

weighting terms 
state vector 
state derivative vector 
state error vector 
output vector 
surface command vector 
pilot roll stick input 

steady state component of pilot roll stick input 
pilot pitch stick input 

steady-state component of pilot pitch stick input 
quantity measure for a category of complexity elements 
scalar, time-varying disturbance uncertainty estimate 
time index 

elapsed time between data points 
vector of body-axis rotational rates 

Subscripts 

0 

a 
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cmd 

err 
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I/O 

k 

LT 
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MD 

NL 

ref 

I x 

X 

X _1 

initial 

adaptive augmentation 
compensator augmentation 
command 

NDI reference model tracking error 
final 

input / output 

tunable parameter 

lookup table 

MRAC reference model 

mode discrete 

nonlinear 

reference model 

integration 

differentiation 

inversion 

Superscripts 

max 

min 

maximum available control input 
minimum available control input 

Introduction 


This report represents a proposed set of metrics to quantify the relative complexity of 
multiple flight control system designs and for evaluating changes in pilot workload due to the 
behavior of an adaptive controller. Over the past 50 years, flight research has demonstrated that 
adaptive flight controls can be an effective technology for improving aircraft safety and 
performance in the presence of uncertain and changing environments (refs. 1-9). However, the 
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nonlinear, time-varying nature of adaptive algorithms continues to challenge traditional methods 
for the verification and validation of safety-critical flight control systems. 

In April of 2009, The National Aeronautics and Space Administration’s (NASA)’s Integrated 
Resilient Aircraft Control (IRAC) project disseminated a request for information (RFI) to the 
adaptive controls community seeking ideas for potential flight experiments (ref. 10). A workshop 
was held in Chicago in August of 2009 with representatives from industry, academia, and other 
government agencies to discuss the wide variety of RFI responses received by NASA. Three 
focus areas were identified through this process. 

1. Simple, yet effective, adaptive control algorithms should be investigated to help address 
the issue of verification and validation of adaptive flight controls to a safety-critical level. 

2. The appropriate level of pilot awareness and interaction with adaptive control systems 
should be studied. 

3. Techniques should be matured for incorporating feedback information, both static and 
dynamic, from the structure of an aircraft into the flight control system to ensure structural 
integrity during adaptation. 

The first two of these focus areas prompted the development of a low complexity, 
textbook-like direct model reference adaptive control (MRAC) scheme (ref. 11) for the NASA 
Full-Scale Advanced Systems Testbed (FAST) (refs. 8-9). FAST is a highly modified F-18 
(McDonnell Douglas, now The Boeing Company, Chicago, Illinois) aircraft that contains a 
research flight control system capable of housing advanced flight controls experiments. 
Full-scale piloted in-flight experimentation has been shown to be an effective method for 
uncovering implementation issues of adaptive systems (refs. 1, 4-9). A philosophy of “simpler is 
better” motivated the development of the low complexity MRAC formulation under the 
assumption that simplification leads to lower implementation and verification costs, and 
ultimately to a greater likelihood of acceptance by aircraft manufacturers, operators, and 
certification authorities (refs. 12-13). Reduced complexity can also lead to a safer design by 
reducing the potential for software implementation errors and unexpected interactions, both 
within the aircraft systems and between the controller and the human pilot (refs. 14-17). 

Pilot interaction with adaptive systems can be either beneficial or have unwanted 
consequences. Beneficial interactions include decisions made by the pilot based upon 
information regarding the performance and status of the system. For example, the pilot may 
configure the controller specifically to adapt to a failed control surface, poor pitch damping or for 
severe cross-axis coupling. While altering the closed-loop dynamics of the aircraft to 
accommodate failures or damage, adaptive control also has the potential to adversely affect the 
handling qualities and pilot-in-the-loop oscillation (PIO) characteristics of the system. The 
potential for adverse interactions is compounded by simultaneous changes in piloting technique 
as the pilot also adapts to the altered aircraft’s flight characteristics. Additional adverse 
interactions between the pilot and adaptive controller include increased workload for the pilot 
and nuisance side effects caused by the adaptation. 

The purpose of this report is to suggest potential metrics that will aid the designer in 
selecting and evaluating an appropriate adaptive control design for their application. Other 
important metrics not addressed in this report include measures of the system’s robustness and 
performance, which have been addressed previously by other authors (ref. 18). 


3 


Complexity Metric 

Any flight control design theory must buy its way onto an aircraft by meeting performance 
and robustness specifications without being overly burdensome to design, implement, and 
qualify for flight. Control theories that are complicated to implement and difficult to prove the 
safety of will be bypassed in favor of those that may not perform as well, but can meet the 
project’s cost and schedule requirements. Such a trade study requires the ability to evaluate a 
priori the difficulty in bringing a particular control design to flight. By documenting the 
implementation complexity for future researchers, these metrics begin to construct a 
knowledge-base of experience for the controls community. 

Traditionally, metrics used for evaluating control methods are often nebulous in nature and a 
result of “tribal knowledge” within the controls community. The necessity of full-state feedback, 
the number of generated controller states, and the difficulty of gain scheduling are some 
examples of potential drawbacks that are often cited when considering various modern control 
methods. Implementation of adaptive control is relatively new to the flight controls community, 
and engineers may not have the requisite experience to draw from when making a 
determination of an adaptive algorithm’s suitability for a given application. 

Formal complexity metrics are generally designed for application to the software 
implementation of a system (ref. 19). Examples include the Halstead complexity metrics, which 
include concepts such as volume, difficulty, and effort derived from the number and variety of 
operators and operands in a software program (ref. 20). However, no attempt is made within 
these metrics to differentiate between the relative complexity of different operations. The 
cyclomatic complexity measures of McCabe are based upon the number of linearly independent 
paths through a software program (ref. 21). Neither of these approaches accounts for 
differences in complexity between different elements of control system design. 

Complexity metrics can be used to characterize the impacts of theory selection on the costs 
of designing, implementing, and flight qualifying a control system. The metrics are designed to 
evaluate both fundamental elements of the control theory and any additional elements that, 
while required for implementation, are often omitted from theoretical descriptions. While 
motivated by the need to evaluate adaptive control theories, these metrics are applicable to any 
control theory. 


Quantitative Complexity Metric Description 

Quantitative metrics provide an objective means for comparison between competing control 
methodologies prior to selection and can measure the impact of modifications to the complexity 
of the algorithm during implementation. A proposed quantitative metric for measuring the 
complexity of a control system is given in equation (1). The metric is similar in approach to 
Halstead’s complexity metrics (ref. 20) in that it derives a measure of complexity by counting the 
number of different control system elements used in the design. The proposed method differs 
from Halstead’s primarily in that it accounts for variations in complexity between different types 
of elements. 


J = Wi n I/0 + w 2 (nj x + n x + n x -i + n NL ) + w 3 n k + w 4 n LT + w s n MD (1 ) 

The scalar J is a measure of complexity for the controller. Each component of the control 
design is categorized and tallied, and category totals represented by the series of variables 
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n e E + . Weighting terms w 1 ••• w 5 e R + are used to adjust for inherent differences in the impact 
to complexity of the various component categories. 

Relative magnitudes for the weighting terms are not proposed in this report. Their 
determination will require a rigorous and detailed tracking of resource utilization during multiple 
control law development programs. The complexity metric function in equation (1) is a linear 
approximation that does not contain element interdependencies and other non-linear 
relationships such as those of the Halstead metrics (ref. 20). For example, an increase in the 
number of tunable parameters may cause the complexity of the controller to rise at a 
faster-than-linear rate due to their inter-dependencies. Identification of any higher-order terms 
would be subject to the same empirical investigation as the weighting terms. 

Input and Output Plane Dimensionality (n //0 ) 

The dimensions of the input and output planes impact control system development in 
several ways. Linear robustness analysis is typically performed at each feedback loop and each 
controller output. Noise, latency and sampling rate characteristics for each input and actuator 
dynamics for each output, must be identified and modeled. During testing, the failure modes of 
each input and output must be understood and the effects evaluated. Some examples of the 
importance of understanding possible failure modes of feedback signals include the crashes of 
the NASA X-31 (Enhanced Fighter Maneuverability (EFM) demonstrator) (ref. 22) and Air 
France A330-200 Flight 447 (ref. 14), both cases in which Pitot tube icing was a contributor. 

Integrators, Derivatives, Inverses and Nonlinear Elements (nj x ,rix,n x -i,n NL ) 

Integrators introduce complications to the control system primarily through the need to 
ensure their proper initialization on startup and during mode changes. Examples of incidents 
caused by improper handling of integrators include the premature retirement of NASA’s 
Demonstration for Autonomous Rendezvous Technology (DART) spacecraft (ref. 23), and the 
mission interruption of the joint NASA and European Space Agency (ESA) Solar Heliospheric 
Observatory (SOHO) (ref. 24). Derivatives of signals introduce noise into the system, which can 
ultimately lead to numerical instability and may require filtering. The number of scalar divides 
and matrix inversions add to the complexity of the system by requiring special handling to avoid 
numerical errors. Nonlinear elements such as integrator limits, saturation and rate limits, 
nonlinear mathematical operations, and dead-bands typically complicate linear robustness 
analysis and create additional test scenarios. These four elements share a common weighting 
coefficient because they are estimated to have roughly the same impact on the cost of control 
system development. 

Tunable Parameters ( n k ) 

Tunable parameters are elements of the controller such as gains, thresholds, and filter 
coefficients that can be adjusted to affect controller performance. These parameters tend to 
complicate the design and verification of the controller considerably due to the potential for 
unforeseen interactions between them. The fewer tunable parameters a system has, the easier 
it is for the designer to understand the impacts of adjusting the design to achieve the desired 
result. Tunable parameters also have an impact on the “fly-fix-fly” process of flight testing as 
further adjustments must be made to correct for a lack of model fidelity. In 1998, American 
Airlines flight 1340 crashed just short of the runway due to excessive pitch oscillations caused 
by a gain in the autopilot of the Boeing 727 that was not properly designed for the approach 
speed used by the pilots (refs. 17 and 25). 
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Lookup Table Dimensionality (n LT ) 

A lookup table encodes a function for use in the control system. Examples of look-up tables 
include gain schedules and coefficient tables. This metric captures their total dimensionality by 
summing together the number of independent input variables. Ensuring proper interpolation 
within the boundaries of the table and extrapolation or limiting outside the table complicates the 
testing effort. These types of tables often also include additional complicating features such as 
pre-lookup calculations and index memory for efficient operation. 

Mode Discretes (n MD ) 

Switchable modes within a control system, such as flap and gear settings, and autopilot 
modes require considerable effort to test, which is especially true when it comes to evaluating 
the interactions of multiple modes and the effects of switching from one mode to another. One 
example of the unexpected complexity effects of mode transitions is the crash of TWA Flight 
843, in which the combination of a state transition from ground to flight and a faulty 
angle-of-attack sensor contributed to a false stall warning immediately after takeoff (ref. 16). 

Complexity Comparison between Multiple Adaptive Controllers 

The complexity metrics are applied to three variations of the I RAC experiment model 
reference adaptive controllers (refs. 8-9) designed for the FAST aircraft, and to the Gen-2A 
controller from the Intelligent Flight Control System (IFCS) experiment (refs. 6-7). Both 
experiments were conducted under the NASA Aviation Safety program. The IFCS flight 
research program was conducted on a highly modified F-15 aircraft (McDonnell Douglas, now 
The Boeing Company, Chicago, Illinois) and concluded approximately 2 years prior to the 
beginning of the I RAC flights on the FAST F-18 aircraft. Despite implementation on different 
avionics systems and different aircraft, the two adaptive controls experiments shared a similar 
set of top-level design requirements. Both experiments were designed to evaluate direct 
adaptive model-reference algorithms integrated with nonlinear dynamic inversion (NDI) baseline 
controllers as pilot-initiated emergency systems. The simulated failures used to evaluate the 
controllers were also similar, including a destabilized pitch axis scenario and a failed stabilator. 

Controller Descriptions 

Figure 1 shows the top-level block diagram of the integrated MRAC and NDI controllers from 
the IRAC experiment. Three variations of the MRAC were implemented and tested to explore 
the relationships between adaptive controller complexity and performance, robustness, and 
handling qualities. The three controllers differed from each other by adding successive levels of 
complexity rather than implementing three entirely independent designs. The positive and 
negative effects of the additional complexity could then be studied. The simplest design, 
referred to as sMRAC, consisted of a single adaptive parameter in each of the pitch and roll 
axes, computed using a basic gradient-based update law. A second design, onMRAC, 
increased the complexity of the update law, while the third, onMRAC+, included one additional 
adaptive parameter in each axis. A detailed description of the IRAC controllers is given in 
reference 26. 
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Figure 1. IRAC NDI and MRAC controller architectures. 

The objective of the IFCS research effort was to develop, through flight research, adaptive 
algorithms to enhance control during primary control surface failures and other aerodynamic 
changes resulting from damage. The first flight phase, referred to as Generation 1 or Gen-1 for 
short, focused on an indirect adaptive aerodynamic parameter identification method (ref. 27). 
The second generation, Gen-2, used neural networks to provide direct adaptive command 
augmentation signals. In principle, this approach does not require information on the nature of 
the failure or the extent of the damage. During flight evaluation with simulated failed control 
surfaces, the Gen-2 controller exhibited poor handling qualities. The goal of Gen-2A, shown in 
figure 2, was to modify the Gen-2 control architecture to improve command tracking and 
handling qualities with control surface failures (ref. 7). 
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Figure 2. IFCS Gen-2A controller architecture. 
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Complexity Metric Application 

The complexity metric measures for the IRAC and IFCS controllers are listed in table 1. 
Values are given separately for the pitch and roll axes of each controller. Elements of the 
system that existed solely to facilitate flight testing or accommodate research hardware were not 
included in the determination of complexity. None of the controllers that were evaluated 
contained derivatives or lookup tables, so those categories are eliminated from table 1 in the 
interest of brevity. Integrators and nonlinear elements which contain tunable parameters were 
counted separately from their tunable parameters, and calculations that were used more than 
once were only counted once wherever possible. 

Table 1. Complexity measures. 




sMRAC 

on M RAC 

onMRAC+ 

IFCS Gen-2A 

I/O dimensions 

pitch 

3 

3 

3 

7 

n I/0 

roll 

3 

3 

3 

7 

integrators 

pitch 

5 

5 

6 

1 

n fx 

roll 

1 

1 

2 

1 

inverses 

pitch 

0 

1 

2 

4 

w *- 1 

roll 

0 

1 

2 

4 

non-linear terms 

pitch 

10 

18 

20 

61 

n NL 

roll 

11 

16 

19 

61 

tunable gains 

pitch 

14 

16 

22 

47 

n k 

roll 

12 

14 

20 

47 

mode discretes . .. 

n MD 

both 

3 

3 

3 
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One of the goals of the IRAC adaptive controls experiment was to design three controllers 
with varying levels of complexity. From table 1 it is apparent that the difference in complexity 
between the IRAC controllers is primarily due to the number of non-linear elements and tunable 
parameters. These differences are mostly a result of increases in the complexity of the adaptive 
update laws. 

By four of the six measures in table 1 , all three of the IRAC controllers are less complex than 
the IFCS Gen-2A control law. The key difference in complexity between the IRAC controllers 
and the IFCS controller is the number of inputs, identified by the measure, n //0 . The larger 
number of inputs used by the IFCS controller drives the substantial increases in the number of 
nonlinear elements, n NL , and in the number of tunable parameters, n k . 

The seemingly incongruous decrease in the number of integrators required by the IFCS 
Gen-2A pitch axis when compared to the IRAC controllers is due to the inclusion of the integral 
of pitch rate as a controlled variable in the IRAC architecture. As described in reference 26, the 
influence of the integral of pitch rate is essentially zeroed out within the controller, but related 
calculations were included in the IRAC controller’s complexity measures for completeness. 
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The complexity metric in equation (1) was applied twice to each of the four adaptive 
controllers in table 1, using two different sets of arbitrary weightings. The resulting complexity 
measures are plotted in figure 3 along with handling quality ratings (ref. 28) gathered during 
research flights with simulated failures. All five controllers were evaluated with a failed stabilator. 
The NDI and IRAC controllers were also tested with unstable pitch dynamics, unstable roll 
dynamics, and significant roll-to-pitch coupling; these failures were not included in the IFCS 
project. Handling quality ratings for the NDI and IRAC controllers were taken from references 8 
and 9, while the IFCS controller ratings are previously unpublished. 



Figure 3. Complexity measure versus handling qualities rating for simulated failures. 

The complexity of the NDI controller is defined as a baseline of zero since it does not 
contain an adaptive controller. The complexity metric was first applied using a weighting set of 
unity. The metric was then applied a second time, in which the tunable parameter weighting was 
set to ten to reflect the difficulty of adaptive control tuning, while the input-output weighting and 
mode discrete weightings were both set to five, under the assumption that they are greater 
contributors to complexity than the mathematical elements, but not as significant as tunable 
parameters. 

By either measure of complexity, the three IRAC controllers are significantly less complex 
than the IFCS Gen-2A controller, while providing approximately the same level of handling 
qualities performance in the presence of failures. Although the onMRAC+ controller is the most 
complex of the three IRAC designs, it exhibits the lowest and most compact set of handling 
qualities ratings, indicating that the additional complexity may provide improved performance. 

Pilot Workload Metrics 

Pilot comments during the IRAC research flights indicated that workload played a significant 
role in the pilot’s handling qualities ratings. Traditionally, Cooper-Harper handling qualities 
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ratings (ref. 28) and the PIO rating scales are used to assess pilot workload. The 
Cooper-Harper scale, in particular, requires the pilot to assess the perceived level of 
compensation required to accomplish the desired task. A more qualitative measure of pilot 
workload can be calculated by cross-plotting measures of the pilot’s aggressiveness and duty 
cycle against each other, as proposed by Shepherd (ref. 29). 


Aggressiveness and Duty Cycle Metrics 

Pilot aggressiveness is a measure of dynamic control deflection, defined by equation (2) as 
the time-averaged summation of the unbiased pilot roll and pitch stick inputs during the 
maneuver. The roll and pitch control bias, 8 ap (t ) and 8 ep (t) respectively, is found by forward 
and backward low-pass filtering of the original signal. Filtering the data in both directions 
eliminates any filter-induced phase shift. Each unbiased control measure is normalized by the 
total allowable deflection range of the appropriate control inceptor. 
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The aggressiveness metric proposed in equation (2) differs from that of Shepherd (ref. 29) 
by introducing a subtraction of the bias terms 8 ap (t) and 8 ep (t). This modification avoids the 
errant categorization of steady, non-zero pilot deflections as highly aggressive commands. 
Equation (2) also combines pitch and roll control deflections into a single metric to support the 
analysis of multi-axis maneuvers. 

Pilot duty cycle is a measure of how frequently the pilot is making a significant adjustment to 
his or her input. Duty cycle is calculated by counting the number of peaks per second in both the 
pitch and roll control stick inputs. A peak is arbitrarily defined as a change in the signal 
magnitude greater than 0.5 percent of the inceptor’s total displacement range and in the 
direction opposite of the signal’s change identified in the previous time step. Two test pilots, 
referred to as Pilot A and Pilot B, participated in the I RAC adaptive controls experiment. Figure 
4 shows example results of the peak detection algorithm for the pitch stick inputs of Pilot A 
during a 2g air-to-air tracking maneuver with a simulated reduced pitch damping failure and the 
sMRAC controller. 



Figure 4. Duty cycle peaks for Pilot A pitch inputs during 2g air-to-air tracking with a reduced 
pitch damping failure and sMRA. 
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This calculation of duty cycle diverges from the method proposed by Shepherd (ref. 29) in 
that it is a determination of the frequency with which the pilot reverses control direction, whereas 
Shepherd measures the percentage of duration of the maneuver during which the pilot’s 
compensation is non-steady. Application of the non-steady compensation technique to the I RAC 
flight data failed to provide sufficient distinction between the different controllers as the pilots, 
regardless of the control system, tended to almost continuously vary their control inputs. 
Significant differences were seen, however, in the frequency of control motion reversals 
between controllers. 

Figure 5 shows a plot of pilot aggressiveness versus duty cycle for two pilots during 2g 
air-to-air tracking maneuvers with a simulated reduced pitch damping failure. The metrics were 
computed for the IRAC non-adaptive NDI controller with no failures or damage, for the NDI 
controller with a simulated failure and for all three IRAC adaptive controllers with the same 
simulated failure. When no failure is present (Unfailed NDI), both Pilot A and Pilot B exhibit very 
similar workload metrics. Following the failure (NDI), with no adaptation, both pilots experience 
an increase in the aggressiveness and duty cycle of their control inputs, although the workload 
increase for Pilot B is significantly higher than that of Pilot A. The simplest adaptive controller 
(sMRAC) restores the duty cycle of Pilot A to approximately the same value as that of the 
unfailed scenario. The two controllers with higher level of complexity (onMRAC, onMRAC+) 
have almost no effect on duty cycle but reduce the aggressiveness of Pilot A, with the most 
complex controller (onMRAC+) displaying nearly the same workload metrics as the unfailed 
case. 



Duty cycle, peaks per second 
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Figure 5. Aggressiveness versus duty cycle metrics for Pilot A and Pilot B during 2g air-to-air 
tracking with a reduced pitch damping failure. 

The workload of Pilot B is also reduced with adaptive control in the presence of the failure, 
but in this case both aggressiveness and duty cycle are affected. There is no significant 
difference between the three adaptive controllers on workload for Pilot B. It is interesting to note 
that even with adaptive control, the workload of Pilot B is much higher than in the unfailed 
scenario. One possible explanation is that despite the improvement in reference model tracking 
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attributable to adaptive augmentation, the gain of Pilot B remains elevated from his initial 
response to the failure. Consequently, even after the adaptive controller restores good pitch 
damping, Pilot B continues to operate at a higher workload than is necessary. This situation is a 
case where information provided to the pilot on the status of the adaptive controller might prove 
beneficial. 


Nuisance Rating Scale 

A nuisance rating scale was developed as part of the Autonomous Collision Avoidance 
Technology (ACAT) project. The nuisance scale, shown in figure 6, is designed to categorize 
the nuisance effects of an autonomous system using a format similar to that of the 
Cooper-Harper rating scale for handling qualities (ref. 28) and the PIO rating scale (ref. 30). An 
adaptive flight control system, while restoring good aircraft handling qualities in the event of a 
failure or damage, may exhibit undesirable side effects ranging from annoying to unsafe. 
Application of the nuisance rating scale provides a consistent, systematic framework for the 
classification of undesirable side effects caused by adaptive flight control systems. 



Figure 6. Nuisance rating scale for autonomous systems on piloted aircraft. 
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During the MRAC research flights, the nuisance scale was initially applied during tests of the 
adaptive controllers with no simulated failures present. The absence of simulated failures 
allowed for a clearer identification of any nuisance characteristics inherent in the controllers. 
Pilot A was the only pilot to apply the nuisance scale to all four controllers with no failures during 
the 2g air-to-air tracking task. Pilot A gave the NDI controller a nuisance rating of 1, which 
served as a baseline for evaluation of the adaptive controllers. The onMRAC+ controller was 
rated as a 4 on the nuisance scale by Pilot A due to objectionable characteristics in the pitch 
axis. The following are excerpts from Pilot A’s comments on 2g air-to-air tracking with the 
onMRAC+ controller and no simulated failures: 

A lot more bobble in gross acquisition than I would have expected . . . had to back out of 
the loop a little bit when doing the gross acquisition, let everything kind of die out; it dies 
out relatively quickly, but little bit bigger oscillations than I would have thought . . . might 
have been a little sensitive in pitch for the gross acquisition and that may have been what 
caused some of the stuff. 

As described in reference 8, an adverse interaction occurred during this maneuver between 
the pilot and the onMRAC+ controller involving the d q adaptive parameter. A minor PIO 
resulted, for which a simple corrective adaptive parameter update law modification term has 
subsequently been proposed (ref. 8). The time-varying pitch sensitivity of this controller was 
also revealed in the Cooper-Harper handling qualities ratings and PIO rating scale evaluations 
by Pilot A, making the nuisance scale a consistent metric, but in this particular case potentially 
redundant. Although no qualities were discovered for any of the MRAC controllers through the 
use of the nuisance scale that weren’t also revealed by the other rating systems, this may be 
more a function of the nature of the controllers’ behavior than a weakness of the nuisance scale 
itself. 


Summary 

Realization of the objectives for the I RAC experiment necessitated the development of 
non-traditional evaluation metrics. A complexity metric was created for measuring progress 
toward a simple yet effective adaptive control method. Control system complexity metrics can 
be useful when selecting a design method, or when making modifications to an existing design, 
in order to better account for the lifecycle impacts of complicating controller structures or 
features. However, significant work remains in determining, for the complexity metric presented 
here, the appropriate term weightings and potential applicability of higher-order term 
combinations. 

Application of the complexity metric to the three I RAC adaptive controllers confirmed the 
expected trend of increased complexity with incremental design enhancements. A significant 
reduction in complexity was demonstrated between all three I RAC designs relative to an earlier 
similar adaptive controller, the IFCS Gen-2A, validating the I RAC project objective of achieving 
simplified adaptive controls. 

Pilot workload metrics were developed and refined to better understand the interactions 
between the pilot and the adaptive controller. Workload metrics can help the control system 
designer evaluate the impact of control system performance changes, and better understand 
pilot handling qualities evaluations. Pilot workload metrics for aggressiveness and duty cycle in 
the presence of a simulated aircraft failure were computed and cross-plotted for flight data from 
three I RAC adaptive controllers and a non-adaptive baseline controller, as well as for the 
baseline controller with no simulated failure. The metrics indicated that the adaptive controllers 
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reduced the pilot workload under a failure scenario, but the degree of reduction was different 
between the two pilots. 

A novel nuisance scale metric was applied to adaptive controllers with no simulated aircraft 
failures or damage. The metric successfully identified a low-level pilot-in-the-loop oscillation as a 
nuisance. The results warrant further application and evaluation of the nuisance scale as a 
pilot-interaction metric for adaptive controllers. 
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